Rose garden promises of intelligent tutoring systems: Blossom or thorn

Intelligent tutoring systems (ITS) have been in existence for over a decade. However, few controlled evaluation studies have been conducted comparing the effectiveness of these systems to more traditional instruction methods. Two main promises of ITSs are examined: (1) Engender more effective and efficient learning in relation to traditional formats; and (2) Reduce the range of learning outcome measures where a majority of individuals are elevated to high performance levels. Bloom (1984) has referred to these as the two sigma problem; to achieve two standard deviation improvements with tutoring over traditional instruction methods. Four ITSs are discussed in relation to the two promises. These tutors have undergone systematic, controlled evaluations: (1) The LISP tutor (Anderson Farrell and Sauers, 1984); (2) Smithtown (Shute and Glaser, in press); (3) Sherlock (Lesgold, Lajoie, Bunzo and Eggan, 1990); and (4) The Pascal ITS (Bonar, Cunningham, Beatty and Well, 1988). Results show that these four tutors do accelerate learning with no degradation in final outcome. Suggestions for improvements to the design and evaluation of ITSs are discussed

Intelligent tutoring systems as tools for investigating individual differences in learning

The ultimate goal of this research is to build an improved model-based selection and classification system for the United States Air Force. Researchers are developing innovative approaches to ability testing. The Learning Abilities Measurement Program (LAMP) examines individual differences in learning abilities, seeking answers to the questions of why some people learn more and better than others and whether there are basic cognitive processes applicable across tasks and domains that are predictive of successful performance (or whether there are more complex problem solving behaviors involved)

Stealth Assessment

An approach to performance-based assessments that embeds assessments in digital games in order to measure how students are progressing toward targeted goals. To succeed in today's interconnected and complex world, workers need to be able to think systemically, creatively, and critically. Equipping K-16 students with these twenty-first-century competencies requires new thinking not only about what should be taught in school but also about how to develop valid assessments to measure and support these competencies. In Stealth Assessment, Valerie Shute and Matthew Ventura investigate an approach that embeds performance-based assessments in digital games. They argue that using well-designed games as vehicles to assess and support learning will help combat students' growing disengagement from school, provide dynamic and ongoing measures of learning processes and outcomes, and offer students opportunities to apply such complex competencies as creativity, problem solving, persistence, and collaboration. Embedding assessments within games provides a way to monitor players' progress toward targeted competencies and to use that information to support learning. Shute and Ventura discuss problems with such traditional assessment methods as multiple-choice questions, review evidence relating to digital games and learning, and illustrate the stealth-assessment approach with a set of assessments they are developing and embedding in the digital game Newton's Playground. These stealth assessments are intended to measure levels of creativity, persistence, and conceptual understanding of Newtonian physics during game play. Finally, they consider future research directions related to stealth assessment in education

Stealth Assessment

An approach to performance-based assessments that embeds assessments in digital games in order to measure how students are progressing toward targeted goals.To succeed in today's interconnected and complex world, workers need to be able to think systemically, creatively, and critically. Equipping K-16 students with these twenty-first-century competencies requires new thinking not only about what should be taught in school but also about how to develop valid assessments to measure and support these competencies. In Stealth Assessment, Valerie Shute and Matthew Ventura investigate an approach that embeds performance-based assessments in digital games. They argue that using well-designed games as vehicles to assess and support learning will help combat students' growing disengagement from school, provide dynamic and ongoing measures of learning processes and outcomes, and offer students opportunities to apply such complex competencies as creativity, problem solving, persistence, and collaboration. Embedding assessments within games provides a way to monitor players' progress toward targeted competencies and to use that information to support learning.Shute and Ventura discuss problems with such traditional assessment methods as multiple-choice questions, review evidence relating to digital games and learning, and illustrate the stealth-assessment approach with a set of assessments they are developing and embedding in the digital game Newton's Playground. These stealth assessments are intended to measure levels of creativity, persistence, and conceptual understanding of Newtonian physics during game play. Finally, they consider future research directions related to stealth assessment in education

Pressed for Space: The Effects of Justification and the Printing Process on Fifteenth-Century Orthography

There is a long-held belief that, prior to the standardisation of written English, printers altered spellings to justify their type. I investigate this claim through an analysis of spelling changes in William Caxton’s two editions of the Canterbury Tales—by examining text within one book, written by one author, and set by one compositor, the only difference between the sections of verse and the sections of prose should be the requirement for justification within the latter. Were the compositors altering spellings to justify their type, we would expect to see a greater number of altered spellings in the prose sections of text. This is not what the results of this study show—instead there is no statistically significant difference between the frequency of spelling changes in justified and non-justified text. However, there is a significantly higher number of abbreviations introduced into the justified text. These results suggest that the compositor of Caxton’s second edition Canterbury Tales did not change spellings to justify his type

Barriers to the uptake and use of feedback in the context of summative assessment

Despite calls for feedback to be incorporated in all assessments, a dichotomy exists between formative and summative assessments. When feedback is provided in a summative context, it is not always used effectively by learners. In this study we explored the reasons for this. We conducted individual interviews with 17 students who had recently received web based feedback following a summative assessment. Constant comparative analysis was conducted for recurring themes. The summative assessment culture, with a focus on avoiding failure, was a dominant and negative influence on the use of feedback. Strong emotions were prevalent throughout the period of assessment and feedback, which reinforced the focus on the need to pass, rather than excel. These affective factors were heightened by interactions with others. The influence of prior learning experiences affected expectations about achievement and the need to use feedback. The summative assessment and subsequent feedback appeared disconnected from future clinical workplace learning. Socio-cultural influences and barriers to feedback need to be understood before attempting to provide feedback after all assessments. A move away from the summative assessment culture may be needed in order to maximise the learning potential of assessments